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Correlation  between  provider  computer  experience  and  accuracy  of  electronic  anesthesia 
charting  -  A  pilot  study  and  performance  improvement  project 

Approximately  33  percent  of  anesthesia  practices  reporting  to  a  national  outcomes 
registry  use  anesthesia  information  management  systems  (AIMS),  a  subset  of  electronic  medical 
records  (EMR)  (Stonemetz  &  Dutton,  2014).  By  contrast,  about  75  percent  of  anesthesia 
departments  in  academic  institutions  use  AIMS  (Anderson  &  Merry,  2015).  However,  less  than 
15  percent  had  capability  to  exchange  such  information  with  other  practices  and  hospitals.  While 
most  American  hospitals  use  at  least  a  basic  form  of  electronic  charting  (Office  of  the  National 
Coordinator  for  Health  Information  Technology,  2015),  increasing  Medicare  reimbursement 
penalties  over  the  next  few  years  will  likely  spur  an  increase  in  the  number  of  practice  settings 
using  a  form  of  electronic  health  record  (Office  of  the  National  Coordinator  for  Health 
Information  Technology,  2013). 

Accuracy  of  these  records  is  an  important  consideration  as  more  of  these  systems  are  put 
to  use  (Kohli  &  See-Lin  Tan,  2016).  Incomplete  or  inaccurate  information  is  a  factor  in  adverse 
events  (Kohli  &  See-Lin  Tan,  2016).  Studies  have  demonstrated  that  adverse  events  lead  to 
longer  hospital  stays  (Forster,  Kyeremanteng,  Hooper,  Shojania,  &  van  Walraven,  2008)  and 
may  lead  to  increased  mortality  of  patients  (Han  et  al.,  2005),  although  these  analyses  did  not 
involve  anesthesia  charting,  but  electronic  records  in  general. 

Ideally,  according  to  Moore  and  Richardson  (2015),  an  AIMS  must  be  free  of  error,  as 
much  as  possible,  and  readable.  The  user  interface  (UI)  should  not  steal  attention  from  rendering 
care.  Its  usefulness  should  include  automation  of  inventory  and  charges  for  equipment  and 
services.  The  aggregate  of  patient  data  generated  should  be  of  a  quality  to  improve  care,  decrease 
adverse  incidents,  and  enhance  research.  Use  of  AIMS  should  improve  day-to-day  and  long-term 


organization  of  staff.  Finally,  monitoring  AIMS  records  should  improve  adherence  to  medical- 
legal  requirements  and  submissions  to  government  monitoring  agencies. 

Recent  studies  of  AIMS  accuracy  (Anderson  &  Merry,  2015;  Avidan  &  Weissman,  2012; 
Davis,  Green,  Colquhoun,  Hage,  &  Biddle,  2012;  Lamer  et  al.,  2015;  Simpao,  Pruitt,  Cook- 
Sather,  Gumaney,  &  Rehman,  2012;  Wilbanks,  2014;  Wilbanks,  Moss,  &  Bemer,  2013)  relied 
on  early  efforts  in  healthcare  IT  from  the  1970s  through  1990s  to  compare  electronic  records 
with  handwritten  charts.  These  were  studies  such  as  Differences  between  handwritten  and 
automatic  blood  pressure  records  (Cook,  McDonald,  &  Nunziata,  1989),  Automated  charting  of 
physiological  variables  in  anesthesia:  A  quantitative  comparison  of  automated  versus 
handwritten  anesthesia  records  (Lerou,  Dirksen,  van  Daele,  Nijhuis,  &  Crul,  1988),  and  Are 
automated  anesthesia  records  better?  (Thrush,  1992),  among  many  others.  Earlier  analyses 
primarily  focused  on  establishing  that  AIMS  were  accurate,  or  at  least  as  much  as  paper  records, 
and  on  provider  vigilance.  Generally,  electronic  charting  was  found  to  be  accurate,  but  in  need  of 
improvement  and  centered  on  analysis  of  recording  physiological  information  from  anesthesia 
monitors  (Wilbanks,  2014).  For  example,  one  early  electronic  system  recorded  blood  pressure 
measurements  20-30  mmHg  higher,  and  more  accurate,  than  those  charted  simultaneously  on 
paper  (Shibutani,  Subhedar,  Shirasaki,  Sakata,  &  Ogawa,  1992). 

In  anesthesia  documentation,  accuracy  and  completeness  are  equally  important.  However, 
a  record  might  be  accurate  in  physiological  readings  (e.g.,  vital  signs  of  heart  and  respiratory 
rate,  blood  pressure,  pulse  oximetry)  but  incomplete  by  professional  standards  (e.g.,  vital  signs 
documented  at  least  every  five  minutes,  notation  of  type  of  anesthesia  and  procedures 
performed).  In  such  a  case,  the  chart  is  accurate,  but  incomplete.  Likewise,  a  record  might  be 
complete  if  it  contains  the  required  data  points,  but  inaccurate  if  significant  time  markers  are 


incorrectly  documented  (e.g.,  the  antibiotic  was  entered  by  a  pre-programmed  template  as  being 
administered  10  minutes  prior  to  entry  into  the  operating  room,  but  was  actually  administered 
much  earlier).  Such  a  record  is  complete,  but  inaccurate.  This  theme  of  accuracy  and 
completeness  is  inherent  in  meeting  the  anesthesia  standard  of  care  for  documentation  (American 
Association  of  Nurse  Anesthetists,  1991;  American  Society  of  Anesthesiologists  Committee  of 
Quality  Management  and  Departmental  Administration,  2013). 

Purpose 

This  performance  improve  project  and  pilot  study  was  intended  to  examine  one  aspect  of 
automated  charting,  that  of  automatic  insertion  of  events  into  an  anesthesia  record  file.  This 
common  practice  is  performed  to  help  anesthesia  providers  remember  to  include  all  important 
events  of  a  case.  However,  the  inserted  entries  must  be  changed  by  the  provider  to  reflect  the 
actual  occurrences  and  times  of  these  events  during  each  case.  Examining  whether  or  not 
providers  were  doing  this,  and  who  was  doing  this,  was  the  purpose  of  this  project. 

Synthesis 

Recent  studies  listed  in  the  review  of  literature  focused  on  accuracy  to  examine  usability 
-  ease  of  use  -  and  effectiveness  of  electronic  charts.  These  questions  focus  more  on 
functionality  and  achieving  a  complete  record  (Wilbanks  et  al.,  2013).  Three  themes  emerging 
from  these  studies  are  correctness ,  usability,  and  developing  technologies,  discussed  as  follows. 

Correctness 

The  concept  of  correctness,  which  included  accuracy,  here  denoting  correct  entry  of  data 
content,  time  stamps  and  similar  information,  either  by  the  provider  or  automatically  from  the 
AIMS,  was  a  term  put  forth  by  Wilbanks  (2014).  Wilbanks  et  al.  (2013)  specifically  noted  that 


completeness  and  ease  of  use  in  electronic  charting  was  greatly  increased  by  programmed  entries 
in  templates.  However,  accuracy  sometimes  suffered  as  a  result  of  human  fallibility. 

Incorrectness  by  omission,  incompleteness.  Another  concept  of  completeness  referred 
to  reporting  of  all  events,  parameters,  procedures,  and  drugs  during  an  anesthetic,  their  inclusion 
being  necessary  to  comprise  a  full  record  of  a  case.  As  the  signatory  of  the  record,  a  provider  is 
responsible  for  its  integrity  (Kohli  &  See-Lin  Tan,  2016).  However,  Wilbanks  et  al.  (2013) 
demonstrated  that  human  omission  was  the  most  frequent  cause  of  incorrect  records.  Likewise, 
Ehrenfeld,  Epstein,  Bader,  Kheterpal,  and  Sandberg  (2011)  noted  that  when  automated  data 
collection  was  incomplete,  or  inaccurate  for  some  reason,  it  was  incumbent  on  the  provider  to 
notate  the  error  and  make  corrections  regarding  which  interventions  were  lacking.  Thus, 
completeness  evolved  as  a  subset  of  the  theme  of  correctness.  The  comprehensive  reporting  of 
all  appropriate  events  during  an  anesthetic  was  lacking,  and  human  error  was  primarily  to  blame 
(Avidan  &  Weissman,  2012;  Ehrenfeld  et  al.,  2011;  Simpao  et  al.,  2012;  Wilbanks  et  al.,  2013). 

Several  reasons  might  explain  why  charts  are  incomplete,  such  as  forgetfulness  while 
charting  after  the  fact  following  a  critical  period  or  event  (Ehrenfeld  et  al.,  201 1).  Anderson  and 
Merry  (2015)  noted  that  data  quality  suitable  for  pooling  of  patient  information,  for  care  or 
research,  would  be  difficult  to  generate  from  an  AIMS  with  an  inefficient,  challenging  user 
interface,  as  such  UIs  contributed  to  inaccurate,  incomplete  data.  The  observers  in  the  Wilbanks 
et  al.  (2013)  research  noted  a  high  degree  of  accuracy,  but  fallible  provider  input  with  the  use  of 
case  templates  to  populate  appropriate  events  to  a  chart,  which  must  then  be  fine-tuned  to  match 
reality.  Anderson  and  Merry  (2015)  highlighted  templates  as  the  cause  of  errors,  making  AIMS 
less  reliable  and  emphasizing  the  responsibility  of  providers  to  ensure  records  were  complete  and 
correct.  This  was  not  necessarily  seen  in  early,  basic  AIMS  versions,  when  providers  had  to 


generate  manual  entries  without  the  aid  of  automatic  entry  of  events  by  pre-programmed 
templates.  The  same  study  specified  poor  interface  design  as  a  primary  cause  of  inadequate  or 
incorrect  data,  lessening  its  value  for  forensic  use  (Anderson  &  Merry,  2015). 

Detecting  and  correcting  inaccurate  records.  In  order  to  detect  error,  it  must  be 
observed.  Researching  the  accuracy  of  records  was  accomplished  in  several  different  ways. 
Researchers  approached  the  problem  with  post-hoc  observation  and  concurrent  observation. 
When  they  attempted  to  correct,  in  addition  to  detecting,  the  problems,  they  used  active,  real¬ 
time  intervention,  or  after-the-fact  alteration  of  records. 

Post-hoc  observation.  Lamer  et  al.  (2015)  and  Marian,  Bayman,  Gillett,  Hadder,  and 
Todd  (2016)  reviewed  finished  records  though  only  Lamer  and  colleagues  sought  to  correct 
them.  In  the  French,  Lamer  et  al.  study,  patient  records  were  altered  in  the  AIMS  database,  with 
no  discussion  of  the  implications  of  such  action  in  the  context  of  U.S.  FDA  regulations.  In  the 
Iowa  study  by  Marian  et  al.,  alarms  were  triggered  by  the  observation  that  university  hospital 
patients  in  the  year  following  AIMS  installation  showed  a  6.1 -percent  increase  in  perioperative 
acuity  scores  without  a  change  in  patient  population.  Correction  of  the  AIMS  returned  the 
university  health  system  to  values  in  keeping  with  U.S.  national  averages. 

Concurrent  observation.  In  several  cases,  researchers  placed  an  observer  in  the  operating 
room  to  record  events  for  later  comparison  with  the  chart  developed  by  the  anesthesia  provider 
(Davis  et  al.,  2012;  Simpao  et  al.,  2012;  Wilbanks  et  al.,  2013).  Others  used  the  AIMS  as  the 
observer  (Avidan  &  Weissman,  2012;  Ehrenfeld  et  al.,  2011).  With  the  AIMS'  real-time 
surveillance,  researchers  had  installed  preprogrammed  interventions  to  enhance  completeness  of 
records.  In  the  case  of  Avidan  and  Weissman,  the  researchers  reprogrammed  their  AIMS  to 
present  context-sensitive,  mandatory  entry  prompts  to  anesthesia  providers,  requiring  completion 


of  all  key  items  before  proceeding  (2012).  It  should  be  noted  that  some,  but  not  all,  AIMS 
software  can  be  set  up  with  required-entry  fields,  sometimes  in  a  context-sensitive  manner,  but 
this  varies  widely  depending  upon  each  software  developer.  Further,  as  in  the  French  study 
(Lamer  et  al.,  2015),  Avidan  and  Weissman  (2012)  had  no  need  to  address  the  implications  of 
altering  what  would  have  been,  in  the  United  States,  an  FDA-approved  medical  device  for  their 
purposes.  A  nuanced,  real-time  intervention,  without  rewriting  the  AIMS,  was  applied  by 
Ehrenfeld  et  al.  (2011),  in  which  electronic  observation  of  more  than  10-minute  gaps  in  blood 
pressure  readings  triggered  pager  and  text  message  notifications  to  the  anesthesia  providers, 
which  prompted  them  to  complete  the  chart. 

Incompleteness  of  voluntary  reporting.  Another  critical  factor  of  completeness  is 
reporting  of  events,  especially  critical  events.  Simpao  et  al.  (2012)  observed  995  cases  to 
determine  the  extent  providers  recorded  pediatric  patient  emesis  on  induction  of  anesthesia. 
Reporting  of  such  events  was  low  (three  reports),  and  even  lower  for  quality  improvement 
reports  (one  report),  although  the  event  was  rare  enough  (eight  incidents  observed)  that  it  was 
difficult  to  analyze.  They  concluded  that  such  critical  events  were  not  rigorously  reported  in 
anesthesia  records,  and  even  less  so  in  quality  improvement  reports. 

On  a  more  common  level,  everyday  charting  of  the  details  of  key  anesthetic  events  is  inherent  in 
accurate  charting.  For  this,  Avidan  and  Weissman  (2012)  chose  four  key  forms  and 
reprogrammed  their  AIMS  to  make  those  items  mandatory.  Changing  the  charting  requirements 
from  voluntary  to  mandatory  raised  the  completion  rate  from  the  59-  to  92-percent  range  to  99.6 
percent.  Similarly,  the  active  intervention  of  paging  and  texting  when  gaps  in  charting  blood 
pressure  were  noted  (Ehrenfeld  et  al.,  2011)  increased  completeness  from  about  93  percent  to 


98.5  percent  post-intervention. 


Comparison  of  AIMS  charts  to  handwritten.  On  the  subject  of  accuracy,  Anderson  and 
Merry  (2015)  remarked  on  the  precise  recording  of  electronic  data  compared  to  handwritten 
charts.  Researchers  cited  “smoothing”  -  the  human  tendency  to  avoid  charting  extremes  of 
physiologic  readings,  called  “railroad  tracks”  (see  Appendix.,  top  section).  Electronic  records 
showed  the  more  realistic  variability  of  readings  such  as  blood  pressure  and  heart  rate  noted 
during  an  anesthetic. 

Furthermore,  when  filling  in  the  gaps  on  a  handwritten  record,  providers  do  not  typically  leave 
obvious  blanks.  This  was  illustrated  by  a  natural  experimental  that  served  as  a  control  group  for 
Ehrenfeld  et  al.  (201 1).  In  their  study  of  gaps  in  blood  pressure  documented  in  AIMS  in  three 
facilities,  one  hospital  was  temporarily  unable  to  use  the  AIMS,  which  yielded  500  paper  charts 
for  comparison.  The  authors  noted  that  none  of  the  handwritten  charts  had  any  gaps  in  blood 
pressure  readings. 

Usability 

This  theme  encompassed  a  broad  aspect  of  AIMS  use.  One  aspect  is  providers’  ability  to 
efficiently  use  the  software  (Avidan  &  Weissman,  2012;  Marian  et  al.,  2016).  In  order  to  be 
defined  as  efficiently  usable,  they  must  be  able  to  do  so  without  loss  of  vigilance  (Davis  et  al., 
2012).  Further,  the  records  recorded  must  be  correct  (Anderson  &  Merry,  2015;  Avidan  & 
Weissman,  2012;  Ehrenfeld  et  al.,  2011;  Kohli  &  See-Lin  Tan,  2016;  Lamer  et  al.,  2015;  Simpao 
et  al.,  2012;  Wilbanks  et  al.,  2013)  and  thus  valid  and  available  for  future  reference,  research, 
and  legal  questions. 

User  interface  complexity.  The  UI  was  cited  as  one  of  the  most  common  enhancers  or 
impediments  to  accurate  charting  (Avidan  &  Weissman,  2012;  Davis  et  al.,  2012;  Ehrenfeld  et 


al.,  2011;  Kohli  &  See-Lin  Tan,  2016;  Lamer  et  al.,  2015;  Marian  et  al.,  2016;  Simpao  et  al., 
2012;  Wilbanks  et  al.,  2013).  In  designing  forms  with  fields  that  required  an  entry  before  moving 
on,  Avidan  and  Weissman  (2012)  considered  that  overly  complex  and  deep  menus  to  be  a 
significant  reason  why  providers  entered  incorrect  information,  or  simply  left  entries  blank. 
Further,  the  required  information  presented  had  to  be  appropriate  to  the  circumstances  being 
charting,  or  context-sensitive.  Davis  et  al.  (2012)  tested  the  concern  that  having  to  click  through 
menus  in  order  to  chart  would  be  distracting  to  providers.  Both  studies  vindicated  electronic 
charting  as  highly  accurate,  if  AIMS  were  designed  to  be  context  sensitive,  and  no  more 
distracting  than  paper  charting,  respectively.  However,  Marian  et  al.  (2016)  demonstrated  that 
AIMS  design  does  affect  the  accuracy  of  information  recorded,  in  that  an  overly  complex  set  of 
forms  skewed  preoperative  assessment  data.  In  that  case,  redesigning  and  simplifying  the  forms 
corrected  the  errors. 

Templates.  As  previously  discussed,  relying  upon  voluntary  reporting  of  all  events 
during  an  anesthetic  ensures  that  some  information  might  be  left  out.  An  attempt  to  correct  these 
omissions  was  the  focus  of  Wilbanks  et  al.  (2013)  examination  of  the  use  of  pre-programmed 
templates.  With  these,  a  set  of  events  typically  entered  for  a  procedure  is  imported  into  a  chart 
and  the  provider  must  adjust  times  and  content  to  make  them  accurate.  Automatic  import  greatly 
increased  completeness  of  the  chart,  regarding  inclusion  of  events  appropriate  to  the  context  of 
the  case.  This  study  revealed  human  error  was  the  most  common  cause  of  either  omission  of 
events,  or  failure  to  correct  data  from  imported  events  to  reflect  reality  for  that  case.  If  the 
context-sensitive  templates  were  not  changed,  or  were  not  applicable,  such  templates  were  worse 
than  useless,  as  they  introduced  error  into  the  record  (Anderson  &  Merry,  2015;  Lamer  et  al., 


2015;  Wilbanks  et  al.,  2013). 


Context-sensitive  prompts.  Imported,  preprogrammed  templates  containing  events 
particular  to  each  type  of  case  qualified  as  context-sensitive.  However,  Avidan  and  Weissman 
(2012)  instead  altered  the  UI  on  key  forms  to  present  only  applicable  fields  for  completion.  For 
example,  a  page  detailing  a  general  anesthetic  with  endotracheal  intubation  did  not  include 
mandatory  data  prompts  to  detail  a  spinal  anesthetic.  In  this  way,  provider  satisfaction  with  and 
appropriate  use  of  the  AIMS  was  increased  to  nearly  100  percent. 

Another  aspect  of  the  context-sensitive  UI  was  automated  provider  alerts  such  as  were 
developed  by  Ehrenfeld  et  al.  (201 1).  Anderson  and  Merry  (2015)  listed  utility  of  such  alerts  for 
critical  events,  such  as  hypotension,  or  best-practice  guidelines,  such  as  administration  of 
perioperative  beta  blockers  or  antibiotics.  Patient  data  was  also  used  to  trigger  reminders 
regarding  prevention  of  postoperative  nausea  and  vomiting.  Patient  demographics,  such  as  age, 
was  used  to  present  physiological  alarms,  such  as  for  pediatric  heart  rate. 

Controls  and  unintended  consequences  of  AIMS.  Spurious  charting  of  data  was 
demonstrated  by  Marian  et  al.  (2016)  to  alter  patient  assessment  data  simply  by  virtue  of  an 
overly  complex  AIMS  UI.  In  this  case,  the  errors  may  have  gone  unnoticed  except  for  an 
external  administrative  audit,  which  noted  an  increase  in  the  American  Society  of 
Anesthesiologists  Physical  Status  (ASA  PS)  scores  for  6.1  percent  of  patients  in  a  university 
health  system,  with  no  corresponding  change  in  patient  population.  The  errors  corresponded  with 
the  installation,  one  year  prior,  of  an  AIMS.  Controls  to  prevent  such  errors  were  installed  to 
change  the  provider  workflow,  which  addressed  data  entry.  Visual  presentation  of  the  data  item 
on  the  AIMS  form  enhanced  communication  between  providers.  However,  prevention  of 
spurious  charting  was  not  possible  within  the  AIMS,  and  physical  control  of  changes  to  the  data 


item  was  only  achieved  by  implementing  a  policy  that  only  anesthesia  providers  would  chart  the 
data  item. 

Controls  to  prevent  errors  within  AIMS  were  theoretically  achieved  by  comparison  of 
large  datasets  of  cases  to  establish  probable  norms  for  each  type  of  anesthetic  (Lamer  et  al., 
2015).  However,  such  a  review  achieved  only  the  establishment  a  normal  context,  such  as  the 
finding  that  an  event  for  induction  of  general  anesthesia  was  typically  followed  by  an  event 
describing  placement  of  an  airway  device.  Outliers  from  the  normal  context  may  be  the  cases 
requiring  scrutiny,  and  would  necessarily  fall  outside  the  norm.  This  use  of  proxy  events  (e.g., 
intubation,  extubation)  by  Lamer  and  colleagues  (2015)  allowed  them  to  increase  their 
completion  rate  to  nearly  100  percent  by  generating  events  (start-  and  end  of  anesthesia  care) 
previously  left  incomplete  in  20  percent  of  their  anesthetic  records.  The  authors  indicated  that  the 
lack  of  mandatory  fields  and  a  poor  user  interface  were  possible  reasons  for  their  poor 
completion  rate.  Improving  the  usability  of  their  AIMS  would  improve  completion  rates  and 
obviate  or  reduce  the  need  for  post-hoc  control. 

Developing  Technologies 

With  this  theme,  authors  explored  aspects  of  improvement  in  AIMS  to  achieve  goals 
envisioned  by  a  pinnacle  system  (Anderson  &  Merry,  2015;  Kohli  &  See-Lin  Tan,  2016; 
Wilbanks,  2014).  The  aspects  include  the  mechanical  functions  of  AIMS,  specifically  the  user 
interface  (Avidan  &  Weissman,  2012;  Lamer  et  al.,  2015;  Marian  et  al.,  2016).  Augmentation  of 
clinical  decision  support,  cost  containment,  reduction  of  medication  errors,  improved 
administrative  function,  and  automated  billing  from  AIMS  all  lag  behind  their  potential 
(Anderson  &  Merry,  2015;  Kohli  &  See-Lin  Tan,  2016).  Easy,  fast,  widespread  integration  of 
data  between  AIMS  and  same-hospital  EMRs,  and  among  the  community  of  AIMS,  providers, 


hospitals,  and  healthcare  systems  likewise  require  further  development.  They  have  been 
demonstrated  in  trials,  but  their  use  is  not  yet  widespread  in  the  many  various  AIMS.  Finally,  a 
significant  facet  of  the  theme  of  developing  technologies  was  universality.  Both  Anderson  and 
Merry  (2015)  and  Kohli  and  See-Lin  Tan  (2016)  noted  that  the  hoped-for  uses  of  large  electronic 
datasets  would  not  come  to  fruition  unless  a  standardized  method  of  communication  between  the 
various  AIMS,  government  and  private,  was  perfected. 

Knowledge  Gap 

Currently,  there  is  no  standard  across  anesthesia  records,  much  less  all  electronic 
medical  records.  A  contemporary  example  of  this  problem  is  the  difficulty  experienced  by  the 
Veterans  Administration  health  care  system  merging  their  records  with  those  in  the  Department 
of  Defense.  Although  ordered  by  Congress  to  complete  the  task  years  ago  (Panangala  &  lansen, 
2013),  it  remains  incomplete  as  of  this  writing,  although  a  stopgap  measure  is  in  the  process  of 
being  implemented.  The  International  Organization  for  Terminology  in  Anesthesia,  in 
cooperation  with  the  International  Health  Terminology  Standards  Development  Organisation  and 
other  entities,  are  at  work  to  develop  this  common  language  (Hurrell  et  al.,  2012).  However, 
accuracy  and  completeness  will  likely  suffer  until  this  is  perfected  and  adopted. 

Evidence-based  strategies 

Although  the  premise  of  the  Fort  Benning  project  was  not  supported  -  at  least  in  the  low 
number  of  cases  reviewed  -  by  evidence  with  statistical  significance,  the  review  of  literature  on 
the  subject  of  accuracy  of  electronic  anesthesia  records  did  reveal  multiple  strategies  that  are,  or 
should  be,  undertaken  by  the  community  for  improvement.  Some  of  these  strategies  are  as 


follows. 


System-to-system  communication.  One  of  the  hurdles  described  is  the  technical 
challenge  of  communication  between  systems  with  different  file  structures.  Hurrell  et  al.  (2012) 
described  a  technique  already  in  common  internet  communication,  called  extensible  markup 
language  (XML).  A  prototype  of  such  AIMS  information  sharing  was  a  key  demonstration  by 
Walsh,  Hurrell,  Wu,  Tomeh,  and  Monk  (2009)  using  XML  to  send  intraoperative  data  to  the 
National  Surgical  Quality  Improvement  Program  (NSQIP).  These  researchers  developed  an 
XML  protocol  to  translate  AIMS  records  into  a  file  readable  by  another  system.  Additionally,  to 
ensure  validity  of  the  information,  they  used  nomenclature  developed  with  EHR  interoperability 
as  a  goal,  including  HL7  and  SNOMED  CT. 

The  Walsh  et  al.  (2009)  group  succeeded  in  using  XML  to  communicate  accurately  the 
anesthesia  record  data  as  intended,  a  strategy  known  as  a  “middleware”  interface  between 
systems.  Anesthesiologist  and  informatics  researcher  Donald  Voltz  expressed  hope  that  such 
interfaces  would  speed  the  pace  of  interoperability  development.  “No  one  is  going  to  develop  for 
75  different  EHR  platforms  when  they  come  out  with  some  sort  of  idea,”  said  Voltz.  “But  instead 
if  we  can  develop  into  a  system  that  then  has  its  fingers  or  tentacles  into  all  the  different  EHRs, 
then  . . .  I’m  accessing  the  patient  data  that  I  need”  (Murphy,  2015). 

While  XML  allows  systems  to  talk  in  a  common  format,  its  lacks  the  common  dictionary 
of  healthcare-associated  terms  that  would  make  it  a  complete  solution  for  interoperability 
problems  (Hurrell  et  al.,  2012).  Development  of  such  a  standard  is  underway  with  the  HL7 
clinical  document  architecture  (CD  A).  Among  other  things,  a  CD  A  is  a  structure  of  common 
terminology  that  could  be  used  in  an  XML  information  transfer  between  systems.  This  weakness 
leads  into  the  second  strategy  for  improving  interoperability. 


Person-to-person  communication.  Hurrell  et  al.  (2012)  described  the  use  of  the  CDA  as 
a  method  of  improving  interoperability.  Again,  a  prototype  was  described  (Hirai,  2009),  which 
Hurrell  and  colleagues  said  was  functional.  The  CDA  file  includes  both  person-  and  computer- 
readable  and  writeable  sections.  They  said  the  ability  to  restrict  the  content  to  a  particular 
vocabulary  suited  it  for  use  as  an  interoperability  tool.  However,  it  did  not  eliminate  the 
possibility  that  the  information  might  not  be  interpreted  differently  by  different  providers 
(Hurrell  et  al.,  2012). 

For  this  reason,  Hurrell  et  al.  (2012)  described  efforts  by  anesthesia  providers  and 
information  technology  experts  to  develop  an  overall  anesthesia  dictionary  of  terms.  This  would, 
theoretically,  solve  the  problem  of  semantic  interoperability.  The  IOTA  group,  among  others, 
used  online  submissions  to  augment  the  SNOMED  CT  database  of  terms.  Submissions  included 
descriptions  of  processes  and  cases  in  anesthesia.  Of  importance  was  the  inclusion  of  medical 
equipment  communication  standards,  which  would  enable  correlation  of  data  for  the  same 
parameter  that  might  differ  between  devices.  The  result  was  more  than  5,000  anesthesia  terms, 
processes,  equipment,  drugs,  and  other  information  brought  into  the  previously  anesthesia- 
generic  SNOMED  CT  system. 

The  significance  of  this,  according  to  Hurrell  et  al.  (2012),  was  the  change  in  the  use  of 
the  middleware  interface.  Although  the  XML  interface  allowed  file  systems  to  exchange 
information,  each  system-to-system  connection  had  to  include  an  agreed-upon  standard 
vocabulary.  This  had  to  been  performed  for  each  such  interface  in  order  for  the  information  to  be 
valid  between  the  parties.  With  the  development  of  an  international  standard,  that  vocabulary 
was  already  established.  However,  according  to  Kohli  and  See-Lin  Tan  (2016),  the  difficulty  lies 
in  bringing  together  all  of  the  many  different  standards,  both  public  and  proprietary. 


Furthermore,  individual  settings  often  have  differing  institutional  priorities.  When  achieved, 
Hurrell  et  al.  (2012)  wrote,  this  would  encompass  semantic  standards  for  anesthesia  terminology 
and,  importantly,  physiological  data  shared  between  different  medical  devices. 

Maintaining  information  infrastructure  in  a  complex  system.  Because  the  healthcare 
information  sector  is  complex,  often  proprietary,  and  comprised  of  both  private  and  government 
efforts,  the  integration  of  systems  still  lags  behind  available  technology,  as  Kohli  and  See-Lin 
Tan  (2016)  stated.  Further,  in  a  system  as  complex  as  healthcare,  new  developments  ensure  a 
constantly  changing  environment  (Braa,  Hanseth,  Heywood,  Woinshet,  &  Shaw,  2009).  The 
theory  to  develop  and  maintain  an  information  infrastructure  in  developing  countries  was  also 
proposed  as  a  strategy  to  face  the  complexity  of  health  FT  in  developed  countries.  According  to 
Braa  and  colleagues,  the  flexible  standards  system  they  described  would  apply,  using  complexity 
science  as  a  theoretical  underpinning.  These  researchers  used  complexity  theory  to  develop  a 
solution  to  their  particular  problem  in  electronic  records.  They  proposed  that  their  techniques  and 
use  of  theory  could  be  more  broadly  used. 

Complexity  theory.  Kohli  and  See-Lin  Tan  (2016)  described  the  flexible  standard  as 
allowing  individual  institutions  or  systems  to  maintain  their  own  information  infrastructure,  but 
interface  with  others  using  middleware  software.  In  this  way,  organizations  could  maintain  their 
own  nomenclature,  and  decide  in  which  way  they  would  adapt  to  wider  standards.  Braa  et  al. 
(2009)  said  complexity  theory  describes  this  as  allowing  standards,  an  information  order,  to 
emerge  from  below,  rather  than  be  dictated  from  top  down  in  a  way  that  might  or  might  not  fit 
the  information  standards  of  the  lower  echelons.  In  this  way,  standards  would  slowly  evolve 
from  below  to  be  adopted,  if  appropriate,  among  a  wider  audience  of  systems.  Braa  and 
colleagues  noted  that  intricate  information  networks  will  develop  order,  and  that  their  differences 


will  mean  that  they  work  to  develop  common  networks  between  each  other  and  the  wider 
information  world,  to  evolve  and  adapt. 

Developing  an  infrastructure  from  scratch.  A  concrete  example  of  this  theoretical 
system  was  described  in  the  healthcare  system  of  South  Africa  (Braa  et  al.,  2009),  which  had 
(during  apartheid)  17  health  systems  divided  by  racial  and  ethnic  tribal  black  South  Africa  areas. 
Healthcare  information  standards  were  equally  diverse  and  not  interoperable.  With  democracy,  a 
standard  for  health  data  became  a  political  goal.  To  begin,  two  regions  cooperated  on  a  basic  set 
of  health  data  set,  while  the  overseeing  organization  developed  a  software  application  to 
administer  that  data  set.  The  problem,  not  just  in  South  Africa,  was  that  each  system  had 
different  requirements.  Also,  each  system  had  different,  autonomous  administrators  and  these 
had  to  permit  admission  of  the  new  standards  to  their  facilities.  Braa  and  colleagues  explained 
that  the  method  around  these  problems  was  twofold.  First,  acknowledgement  that  there  would 
never  be  100-percent  agreement  allowed  the  participants  to  settle  on  at  least  a  minimal  set  of 
standards  that  were  acceptable  to  all.  Secondly,  each  individual  system  was  free  to  augment  their 
data  sets  with  as  much  additional  information  as  they  required.  The  authors  said  this  hierarchy  of 
standards  was  the  key  element  of  the  South  Africa  standardization  process. 

After  developing  a  basic  set  of  47  agreed-upon  pieces  of  data,  the  set  was  successfully  expanded 
for  use  in  all  of  the  regions  and  systems.  Braa  et  al.  (2009)  said  its  success  was  partly  because  it 
was  the  first  universal  data  set  available  to  all  in  the  system,  which  was  an  advantage  for  all  to  be 
able  communicate  information,  at  least  at  that  very  basic  level.  Also,  the  software  application 
developed  by  the  government  health  service  provided  a  common  system  for  working  with  the 
data  set.  It  enabled  local  users  to  manage  their  own  healthcare  systems,  which  had  previously  not 


been  available. 


Complexity  theory  attractors  and  indicators.  Subsequently,  as  black  South  Africa 
districts  were  developing  this  basic  set,  two  white  South  Africa  health  systems  were  brought  in, 
making  the  beginnings  of  the  first  national  healthcare  data  set  (Braa  et  al.,  2009).  These  four 
startup  programs  became,  in  the  language  of  complexity  theory,  attractors.  Other  districts, 
observing  the  successful  example  of  gathering,  disseminating,  and  using  the  new  data  sets  for 
healthcare  analytics  for  the  first  time,  overcame  resistance  to  participation.  The  attraction  of  at 
least  a  minimal  information  package,  using  common  vocabulary,  and  having  a  common  software 
system  for  analysis  supported  an  administrative  need.  Three  years  after  initiation,  in  June  2000, 
South  Africa  established  a  national  data  set  of  essential  elements,  and  by  2005  included 
indicators ,  such  as  nationwide  immunization  rates,  which  are  drawn  from  multiple  data 
elements.  All  of  the  participants  still  maintained  their  own  sets,  expanded  as  fit  their  needs,  but 
all  gave  the  government  at  least  those  basic  elements  to  be  shared.  Additional  elements  have 
since  been  developed  and  added. 

Braa  et  al.  (2009)  said  the  constant  debate  over  indicators  in  the  national  indicator  dataset 
(NIDS)  continues  and,  in  fact,  stimulates  further  change.  An  important  facet  of  flexible 
standardization  is  that  there  is  never  a  finished  data  set.  Also,  the  common  software  tool 
simplifies  information  handling  for  the  participants,  making  changes  easier  to  develop  and 
implement.  Development  of  middleware,  also  called  gateway,  software  has  further  supported 
expansion  and  adaptability  of  the  data  sets. 

These  three  key  studies  -  Walsh  et  al.  (2009),  Hurrell  et  al.  (2012),  and  Braa  et  al.  (2009) 
-  outlined  strategies  to  achieve  interoperability,  which  means  that  the  various  different  electronic 
medical  record  systems  can  communicate  with  each  other.  Walsh  and  colleagues’  classic 
prototype  AIMS  communication  standard  proved  that  the  technology  already  existed  in  2009  to 


accomplish  interoperability,  although  the  common  vocabulary  did  not.  Hurrell  and  colleagues 
described  initial  development  of  datasets  for  semantic  interoperability  as  a  basis  for  growth.  Braa 
and  colleagues  described  how  a  basic  set  of  elements  could  become  an  information  entity  with 
the  ability  to  evolve  and  adapt.  But,  they  noted  that  the  system  change  in  the  South  African 
example  was  stimulated  by  a  political  shock  with  the  fall  of  apartheid.  Whether  the  elevation  of 
user  expectations  for  AIMS,  and  EHRs  generally,  described  by  Kohli  and  See-Lin  Tan  (2016) 
remains  to  be  seen. 

The  question  of  accuracy  of  records  cannot  be  separated  from  the  question  of 
completeness  of  records.  Wilbanks  (2014)  noted  one  common  usage  of  the  terms  to  be  “ accuracy 
is  the  correctness  of  data,  and  completeness  is  the  presence  of  data  (2014,  p.  56).”  For  this 
project,  the  theme  of  correctness  included  the  topics  accuracy  and  completeness ,  a  slight 
variation  of  Wilbanks’  usage.  The  project  hypothesis  is  that  greater  provider  computer 
experience  will  make  them  more  likely  to  change  the  preprogrammed  chart  for  accuracy  in 
timing,  for  example,  and  to  add  (or  remove)  events  to  reflect  the  actual  events  of  the  case.  In 
December  2010,  all  Department  of  Defense  (Army,  Navy/Marine  Corps,  Air  Force,  and  others) 
anesthesia  departments  began  a  mandated  rollout  of  the  Innovian  Anesthesia  AIMS  software  by 
Draeger  Medical,  Inc.  For  most  providers  at  the  Fort  Benning  institution,  this  was  their  first  use 
of  an  AIMS. 

Regarding  the  project  hypothesis,  several  studies  addressed  AIMS  usage.  In  related 
studies,  Deshur  and  Fevine  (2015)  theorized  that  providers’  comfort  level  with  computers 
influenced  the  level  of  distraction  caused  by  the  anesthesia  information  management  system 
(AIMS).  Peterfreund  et  al.  (2011)  attributed  their  92%  increase  in  documentation  of  outcome 
events  with  use  of  AIMS,  in  part,  to  the  relative  youth  and  familiarity  with  technology  in  their 


studied  group  of  providers.  Driscoll,  Columbia,  and  Peterfreund  (2007)  attributed  incomplete 
charting  with  AIMS  to  dependence  upon  free-text  entries,  something  that  preprogrammed 
templates  are  intended  to  avoid  (but  are  not  able  to  avoid  with  the  Innovian  system  -  free  text  is 
required).  As  early  as  1997,  one  study  (Gibby)  identified  a  requirement  that  outcomes,  events, 
and  other  data  be  standardized  or  accurate  outcome  studies,  and  risk/cost  versus  benefit  analyses 
would  not  be  possible.  Sockolow,  Weiner,  Bowles,  and  Lehmann  (2011)  noted  user  satisfaction 
with  new  EMRs  improved  usage. 

Wilbanks  et  al.  (2013)  identified  several  areas  with  common  inaccuracies  in  AIMS. 
Among  these  was  use  of  preprogrammed  templates  to  chart  events  before  they  occur,  intended  to 
prompt  anesthesia  providers  to  enter  all  of  the  necessary  events  for  a  particular  case.  These 
investigators  noted  that  use  of  the  templates  greatly  increased  the  completeness  of  the  AIMS 
charting,  but  created  sources  of  inaccuracies  (such  as  with  time  stamps  and  content).  This 
evidence  is  the  same  as  the  thought  process  that  began  this  process  improvement  project. 

Method 

The  Innovian  system  at  the  Army  community  hospital  on  Fort  Benning,  Georgia,  is 
integrated  with  six  Draeger  Apollo  and  two  Draeger  Fabius  Tiro  anesthesia  machines  and  Philips 
Intellivue  vital  signs  monitors.  It  is  not  integrated  with  other  EMRs  in  use  at  that  facility  for 
charting  outside  of  the  anesthesia  department.  It  is  only  accessible  by  the  anesthesia  service  and, 
in  a  read-only  mode,  by  any  user  with  access  to  the  finished  anesthesia  case  files  loaded  into  the 
general-use  EMR  (Essentris)  at  the  end  of  each  case.  When  the  network  is  functioning,  the  cases 
are  saved  to  a  central  database.  When  the  network  is  down,  cases  are  saved  on  the  individual 
anesthesia  workstations,  then  uploaded  to  the  database  when  the  network  is  reestablished.  The 
Innovian  AIMS  workstations  are  located  in  all  (six)  main  operating  rooms,  the  labor  and  delivery 


OR,  urology  OR,  and  one  in  the  PACU  for  post-case  charting  and  uploading  the  files  to  the 
Essentris  EMR.  The  AIMS  is  not  used  in  the  endoscopy  suite,  interventional  radiology,  or  other 
areas  outside  of  the  OR. 

Standardized  templates  were  developed  from  several  hundred  sample  templates  provided 
in  January  201 1  from  the  DoD  testing  facility,  a  naval  station.  At  Fort  Benning,  a  volunteer 
group  of  four  CRNAs  and  anesthesiologists  developed  guidelines  for  the  templates 
(“environments”)  loaded  into  each  case,  pairing  down  to  half  a  dozen,  which  were  analyzed  in 
this  project.  These  were  a  an  automatically  loaded  global  template  (which  loaded  items  common 
to  all  cases),  and  one  each  loaded  as  appropriate  by  the  provider  for  general  endotracheal 
anesthesia,  general  with  a  laryngeal  mask  airway,  caesarean  section,  pediatric  general,  or 
monitored  anesthesia  care.  Each  contained  items  and  events  thought  likely  to  be  used  during  one 
particular  type  of  case.  Excess  items  (drug  entries  and  events)  were  intended  to  be  deleted. 
Incorrect  items  were  intended  to  be  altered  to  fit  each  case.  This  was  the  basis  for  this  project  - 
were  providers  altering  the  templates  to  make  their  records  reflect  reality,  and  who  was  most 
likely  to  do  so? 

Participants 

The  database  in  the  Fort  Benning  Army  hospital  provided  a  convenience  sample  of 
anesthesia  providers  who  had  used  the  system  since  installation  of  the  AIMS  in  201 1.  Most 
providers  in  the  database  had  already  transferred  to  other  facilities,  as  is  common  in  military 
service.  Those  currently  working  on  Fort  Benning,  and  those  who  still  had  an  e-mail  address  in 
the  DoD  global  contacts  database  were  sent  a  recruitment  letter  explaining  the  project,  a  consent 
form  explaining  the  project,  and  a  survey  developed  to  measure  satisfaction  with  an  EMR 
(Sockolow  et  al.,  2011),  altered  with  permission  of  the  author.  Of  those,  17  returned  the  survey 


and  signed  consent.  Only  one  did  not  meet  inclusion  criteria  because  he  had  only  10  cases  in  the 
database.  The  author’s  cases  were  excluded  from  data  analysis,  although  they  were  analyzed  first 
to  help  develop  the  workflow  for  the  data  collection  tool.  This  yielded  a  pool  of  518  cases  for 
analysis. 

Of  the  16  participants  analyzed,  they  ranged  in  age  from  32  to  63,  1 1  were  male,  and  12 
were  CRNAs.  One  had  30  years’  experience  with  EMRs  outside  of  Innovian,  and  one  had  14 
years’  work  with  other  AIMS;  the  rest  of  the  providers  had  experience  only  with  Innovian.  Most, 
save  two,  had  five  years  or  less  experience  with  Innovian.  For  81  percent  of  the  participants’, 
their  experience  in  anesthesia  was  eight  years  or  less.  Their  self-rated  computer  knowledge  was 
“average”  for  eight,  “below  average”  for  seven,  and  “above  average”  for  one. 

Design 

A  minimum  of  30  cases  were  analyzed  for  each  of  16  anesthesia  providers  in  a 
retrospective  records  review.  Participants  completed  a  23 -question  survey  to  assess  their  level  of 
satisfaction  with  their  use  of  the  AIMS.  The  survey  included  demographic  questions  of  birth 
year,  gender,  job  title  (CRNA  or  anesthesiologist),  years  of  experience  with  AIMS  other  than 
Innovian  (if  any),  years  of  experience  with  the  Innovian  AIMS,  years  of  anesthesia  work 
experience,  and  self-rated  computer  knowledge  (minimal  to  advanced).  The  23  question  was  a 
free-text  question,  What  worked  well  or  what  are  your  concerns  related  to  the  system?  The  22 
primary  survey  questions  were  answered  on  a  Likert  scale  from  Strongly/moderately/mildly 
disagree  to  Strongly/moderately/mildly  agree  with  no  neutral  point.  Per  Sockolow  et  al.  (2011), 
the  survey  evaluates  provider  opinions  of  EMR  quality  of  design,  logistics  of  data,  processes, 
effects  on  quality  of  care,  tangential  effects,  and  the  adoption  process. 


The  Pearson  product  moment  correlation  was  used  to  compare  dependent,  continuous 
variable  of  the  count  of  total  events  changed  with  continuous,  independent  variables  age,  years  of 
Innovian  experience,  years  of  AIMS  experience,  and  years  of  anesthesia  experience.  The 
assumptions  of  the  data  for  this  parametric  test  included  continuous  variables,  lack  of  outlying 
data  points  to  skew  findings,  and  a  linear  relationship  (called  “homoscedasticity”)  (Statistics 
Solutions,  2016).  The  latter  two  assumptions  could  be  considered,  but  not  assumed  true  until  data 
collection  revealed  the  nature  of  the  data. 

Spearman’s  Rank-Order  correlation  was  used  to  compare  the  dependent  continuous 
variable  of  the  count  of  total  events  changed  with  the  nominal  and  ordinal  level  in  independent 
variables  gender,  job  type,  and  self-rated  computer  knowledge  (on  a  Likert  scale).  Assumptions 
of  Spearman’s  test  variables  include  data  on  at  least  the  ordinal  level  (Laerd  Statistics,  2013).  It 
is  assumed  that  the  relationship  between  the  two  variables  compared  increases  and  decreases  in 
conjunction  with  each  other.  These  assumptions  seemed  appropriate  for  the  Likert  variable  of 
self-rated  computer  knowledge,  but  were  not  fulfilled  for  the  nominal  variables  gender  and  job 
type,  which  were  not  central  to  the  hypothesis. 

Data  Collection 

The  primary  investigator  began  each  review  of  a  provider’s  cases  by  recording  the 
demographic  information  into  an  Excel  spreadsheet,  coded  by  the  participant  research 
identification  number.  Using  the  Innovian  Case  Manager  application,  the  database  was 
examined  for  the  total  number  of  cases  the  provider  had  conducted.  Using  the  Innovian  Recorder 
application,  a  picklist  of  cases  was  developed  to  exclude  all  since  the  date  the  provider  was 
informed  of  the  project  and  signed  the  consent.  The  first  case  encountered  with  that  provider’s 
name  in  the  staffing  list  was  selected  and  examined  to  establish  that  they  were  the  sole  anesthesia 


provider  for  that  case,  excluding  short  breaks.  If  so,  the  event  summary  for  the  entire  case  was 
examined  for  time  entered,  and  whether  content  was  changed.  This  summary  was  compared  to 
the  template  of  preprogrammed  events  for  that  environment  (e.g.,  Surgical  time-out,  SAB  placed, 
Estimated  blood  loss,  No  untoward  anesthesia  outcomes  noted  for  case,  IV  antibiotic  prophylaxis 
administered).  If  there  was  a  difference  in  time  or  content,  or  both,  a  tick  mark  was  made  for 
each  on  the  data  collection  sheet  for  that  event.  The  tick  mark  was  either  vertical  (“Yes”)  or 
horizontal  (“No”),  in  order  to  track  that  the  number  of  tick  marks  matched  the  number  of  cases 
counted.  After  all  of  the  events  for  that  case  were  recorded,  the  record  was  closed  and  the  picklist 
reopened  to  find  that  provider’s  next  case,  or  a  random  case  by  that  provider.  Each  event  of  each 
of  518  cases  (31  to  33  cases  were  counted  for  most  providers)  was  documented  in  this  manner. 
After  recording  each  provider’s  case-event  changes,  the  totals  were  entered  in  an  Excel 
spreadsheet.  To  complete  data  collection,  all  of  the  information  -  survey  answers,  demographics, 
case  total,  event  totals  -  were  transferred  to  SPSS  version  24  for  analysis.  Totals  of  events  were 
derived  from  the  ratio  of  changed  items-to-total  events  examined.  Percentages  of  those  changed 
totals  were  calculated  for  each  provider,  and  for  all  providers  as  a  whole. 

Results 

Survey  of  provider  satisfaction  with  EMR 

Anesthesia  providers  were  generally  positive  about  the  Innovian  AIMS  EMR,  reflected 
by  most  answers  of  mildly-  to  strongly-cigree  for  questions  framed  in  a  positive  manner.  They 
felt  Innovian  was  generally  available,  supported,  and  user  friendly.  It  helped  them  in  their  work, 
was  accurate,  and  contributed  to  patient  safety.  They  felt  it  was  worth  the  effort  to  learn  to  use, 


and  that  clinicians  were  satisfied  with  it. 


Further,  they  generally  answered  mildly-  to  strongly-disagree  for  questions  framed  in  the 
negative  manner.  These  indicated  that  these  anesthesia  providers  generally  disagreed  with 
statements  that  the  system  had  frequent  problems,  patients  had  confidentiality  concerns  about  it, 
the  provider  knew  specifically  that  patient  care  had  been  negatively  affected  by  its  use,  and  that 
part  of  increasing  healthcare  costs  were  because  of  computers. 

Test  of  the  hypothesis 

There  were  no  significant  correlations  noted  between  the  independent  variables  (age, 
gender,  job  title  -  CRNA  or  anesthesiologist  -  years  of  experience  with  AIMS  software  other 
than  Innovian,  years  of  experience  with  the  Innovian  AIMS,  years  of  anesthesia  work 
experience,  and  self-rated  computer  knowledge  -  minimal  to  advanced)  and  the  number  of 
events  altered  (Figures  1  and  2).  Measurement  using  Pearson’s  r  showed  no  correlation  between 
the  number  of  events  altered  (dependent  variable)  and  any  of  the  independent  variables  closer  to 
significance  than  p  =.09  level.  The  independent  variables  (Figure  1)  were  age  (p  =.629), 
experience  with  AIMS  other  than  Innovian  (p  =.09),  experience  with  Innovian  AIMS  ip  =.802), 
and  years  of  experience  in  anesthesia  (p  =.973).  The  hypothesis,  that  more  experienced  software 
and  computer  users  would  be  more  likely  to  change  automated  entries,  was  not  statistically 
significant.  There  was  a  significant  correlation  (r  =  .814,  p  <  .001)  between  independent 
variables  Age  and  ExpAnes  (years  of  experience  working  in  anesthesia),  as  would  be  expected. 

Most  providers  changed  a  limited  number  of  key  events  from  their  pre-programmed 
settings,  while  others  were  infrequently  or  never  changed  (Figure  3).  The  data  collection  sheet 
included  70  events  included  in  all  of  the  pre-programmed  templates.  With  a  minimum  of  30 
cases  analyzed  per  providers,  a  theoretical  maximum  of  2,100  events  could  be  altered.  The 


number  of  events  altered  by  the  16  providers  participating  ranged  from  a  low  of  238  to  a  high  of 
580,  with  a  mean  of  345.75. 

The  most  commonly  altered  events  are  listed  in  Table  1.  The  percentages  listed  indicated 
the  frequency  with  which  anesthesia  providers  altered  these  events  from  their  pre-programmed 
values  for  each  template.  The  number  of  cases  recorded  of  each  template  is  listed,  as  is  the 
number  of  events  changed  that  generated  the  percentage  listed.  For  example,  518  cases  were 
review,  thus  all  of  these  had  all  of  the  events  from  the  Global  template.  However,  only  46  cases 
of  c/sections  were  reviewed,  and  1 1  pediatric  general  anesthetic  templates  recorded.  Thus,  the 
percentage  frequencies  for  altered  events  that  were  particular  to  these  templates  are  based  upon  a 
much  lower  number  of  cases  reviewed.  The  number  of  each  type  of  template  reviewed  is 
recorded  next  to  the  percentage-changed  listing. 


Table  1 


Template  Events  Changed 


Event  changed 

Percentage 
of  total 

altered 

Template 

#  of 

cases, 

actual 

#  of  cases, 
template 

Baby  delivered,  time 

99.2% 

c/s 

45 

46 

APGAR,  5-minute,  time 

96.6% 

c/s 

44 

46 

APGAR,  5-minute,  value 

96.6% 

c/s 

44 

46 

APGAR,  1-minute,  time 

96.6% 

c/s 

44 

46 

APGAR,  1-minute,  value 

96.6% 

c/s 

44 

46 

Spinal,  in  place,  time 

94.3% 

c/s 

43 

46 

Spinal,  in  place,  note 

94.3% 

c/s 

43 

46 

Antibiotics  administered,  type 

93.4% 

Global 

484 

518 

IV  in  situ,  time  (Pedi  GA) 

92.9% 

Pedi  GA 

10 

11 

IV  in  situ,  type  (Pedi  GA) 

92.9% 

Pedi  GA 

10 

11 

Manual  vitals,  immediate  postop,  entry 

90.7% 

Global 

470 

518 

Urine  output,  amounts 

90.4% 

Global 

468 

518 

Uterine  incision,  time 

90.4% 

c/s 

42 

46 

Oxygen  administered,  flow  rate  (MAC) 

90.0% 

MAC 

38 

42 

Estimated  blood  loss,  amounts 

89.4% 

Global 

463 

518 

Oxygen  administered,  time  (c/s) 

84.0% 

c/s 

39 

46 

Oxygen  administered,  flow  rate  (c/s) 

84.0% 

c/s 

39 

46 

Antibiotics  administered,  time 

82.0% 

Global 

425 

518 

IV  placed,  time  (MAC) 

82.0% 

MAC 

34 

42 

IV  placed,  note  (MAC) 

82.0% 

MAC 

34 

42 

Pitocin,  time  administered 

81.0% 

c/s 

37 

46 

Pitocin,  amount  added  to  bag 

81.0% 

c/s 

37 

46 

Oxygen  administered,  time  (MAC) 

80.3% 

MAC 

34 

42 

Epidural,  in  place,  time 

80.1% 

c/s 

37 

46 

Epidural,  in  place,  note 

80.1% 

c/s 

37 

46 

IV  placed,  time  (Pedi  GA) 

78.6% 

Pedi  GA 

9 

11 

IV  placed,  note  (Pedi  GA) 

78.6% 

Pedi  GA 

9 

11 

IV  in  situ,  type  (Global) 

72.3% 

Global 

375 

518 

Turnover  to  surgeon,  time 

71.3% 

Global 

369 

518 

Manual  vitals,  immediate  postop,  time 

63.6% 

Global 

329 

518 

Estimated  blood  loss,  times 

61.9% 

Global 

321 

518 

LMA  placed,  time 

60.5% 

LMA  GA 

114 

188 

LMA  placed,  note 

59.7% 

LMA  GA 

112 

188 

LMA  removed,  time 

56.5% 

LMA  GA 

106 

188 

LMA  removed,  note 

55.7% 

LMA  GA 

105 

188 

Urine  output,  times 

49.9% 

Global 

258 

518 

Foley  catheter  placement,  time 

45.2% 

c/s 

21 

46 

Placenta  delivered,  time 

44.6% 

c/s 

21 

46 

Platelet  count  noted,  value 

44.5% 

c/s 

20 

46 

Allis  test,  note 

38.7% 

c/s 

18 

46 

Event  changed 

Percentage 
of  total 

altered 

Template 

#  of 

cases, 

actual 

#  of  cases, 
template 

Surgical  timeout,  time 

30.6% 

Global 

159 

518 

Temperature  measured,  values 

29.8% 

c/s 

14 

46 

IV  in  situ,  time  (Global) 

19.2% 

Global 

99 

518 

Airway  suctioned  before  extubation,  time  (Pedi  GA) 1 

16.7% 

Pedi  GA 

2 

11 

Patient  makes  purposeful  movement,  note  (Pedi  GA) 1 

16.7% 

Pedi  GA 

2 

11 

Bite  block  inserted  before  extubation,  time  (Pedi  GA) 1 

16.7% 

Pedi  GA 

2 

11 

Allis  test,  time 

10.1% 

c/s 

5 

46 

Bite  block  inserted  before  extubation,  type  (Pedi  GA) 1 

9.5% 

Pedi  GA 

1 

11 

Airway  suctioned  before  extubation,  time  (GA) 

5.9% 

GA 

14 

231 

Bite  block  inserted  before  extubation,  type  (GA) 

3.9% 

GA 

9 

231 

Bite  block  inserted  before  extubation,  time  (GA) 

3.6% 

GA 

8 

231 

Anesthesia  start,  time 

3.1% 

Global 

16 

518 

Patient  makes  purposeful  movement,  note  (GA) 

2.7% 

GA 

6 

231 

Anesthesia  end,  time 

2.1% 

Global 

11 

518 

Patient  spontaneously  breathing,  time 

1.3% 

Global 

7 

518 

Anesthesia  start,  note 

0.8% 

Global 

4 

518 

Patient  makes  purposeful  movement,  note 

0.8% 

LMA  GA 

2 

188 

Surgical  timeout,  note 

0.6% 

Global 

3 

518 

Regional  block  assessed  prior  to  incision,  time 

0.6% 

c/s 

0 

46 

Outcome  occurrence,  time 

0.4% 

Global 

2 

518 

Outcome  occurrence,  note 

0.2% 

Global 

1 

518 

Airway  suctioned  before  extubation,  time  (c/s) 1 

0.0% 

c/s 

0 

46 

Patient  makes  purposeful  movement,  note  (c/s) 1 

0.0% 

c/s 

0 

46 

Bite  block  inserted  before  extubation,  time  (c/s) 1 

0.0% 

c/s 

0 

46 

Bite  block  inserted  before  extubation,  type  (c/s) 1 

0.0% 

c/s 

0 

46 

Regional  block  assessed  prior  to  incision,  note 

0.0% 

c/s 

0 

46 

Patient  makes  purposeful  movement,  note  (MAC) 1 

0.0% 

MAC 

0 

42 

Bite  block  inserted  before  extubation,  time  (MAC) 1 

0.0% 

MAC 

0 

42 

Bite  block  inserted  before  extubation,  type  (MAC) 1 

0.0% 

MAC 

0 

42 

Airway  suctioned  before  extubation,  time  (MAC)  1 

0.0% 

MAC 

0 

42 

Note:  Global  template  events  are  entered  into  all  cases.  Listed  also  are  pre-programmed  events 
from  the  user- selectable  templates.  These  are  automatically  entered  into  anesthesia  charts 
depending  upon  while  template  is  selected.  Events  are  listed  in  descending  order  of  those  most 
frequently  altered  of  all  providers’  entries,  as  a  percentage  of  total  cases  for  each  template. 
Superscript  notation  1  indicates  entries  for  general  endotracheal  anesthesia  (purposeful 
movement,  bite  block  usage,  airway  suctioning)  in  cases  that  would  not  typically  involve  GET  A 


(c/section,  MAC,  or  Pedi  GA  with  a  facemask  anesthetic),  which  were  included  on  the  templates 


to  facilitate  charting  if  such  cases  had  to  be  converted  to  GETA.  No  such  case  conversions  were 
noted  during  data  collection. 
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Age 

ExpOutsideAl 

MSYears 

ExpInnovianA 

IMS 

ExpAnes 

TotalEventsC 

ount 

Age 

Pearson  Correlation 

1 

.319 

.086 

.864" 

-.131 

Sig.  (2-tailed) 

.228 

.752 

.000 

.629 

N 

16 

16 

16 

16 

16 

ExpOutsideAlMSYears 

Pearson  Correlation 

.319 

1 

-.272 

.448 

.438 

Sig.  (2 -tailed) 

.228 

.309 

.082 

.090 

N 

16 

16 

16 

16 

16 

ExpInnovianAIMS 

Pearson  Correlation 

.086 

-.272 

1 

-.105 

-.068 

Sig.  (2-tailed) 

.752 

.309 

.699 

.802 

N 

16 

16 

16 

16 

16 

ExpAnes 

Pearson  Correlation 

.864** 

.448 

-.105 

1 

-.009 

Sig.  (2-tailed) 

.000 

.082 

.699 

.973 

N 

16 

16 

16 

16 

16 

TotalEventsCount 

Pearson  Correlation 

-.131 

.438 

-.068 

-.009 

1 

Sig.  (2-tailed) 

.629 

.090 

.802 

.973 

N 

16 

16 

16 

16 

16 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 


Figure  1 .  Correlation  significance  table  showing  relationship  between  dependent  variable 
{TotalEventsCount)  and  ratio-level  independent  variables  {Age,  ExpOutsideAIMSY ears, 
ExpInnovianAIMS,  ExpAnes ).  The  only  significant  correlation  was  between  anesthesia  providers’ 
age  and  their  years  of  anesthesia  experience  {ExpAnes).  Pearson’s  r  test  was  used  to  analyze 
correlation  between  the  parametric  variables  (Polit  &  Beck,  2012). 
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Total  Eve  ntsC 
ount 

Gender 

Job 

Self-rated 

computer 

knowledge 

Spearman's  rho  TotalEventsCount 

Correlation  Coefficient 

1.000 

.132 

.125 

.193 

Sig.  (2 -tailed) 

.627 

.644 

.474 

N 

16 

16 

16 

16 

Gender 

Correlation  Coefficient 

.132 

1.000 

.234 

-.246 

Sig.  (2 -tailed) 

.627 

.384 

.358 

N 

16 

16 

16 

16 

Job 

Correlation  Coefficient 

.125 

.234 

1.000 

.053 

Sig.  (2-tailed) 

.644 

.384 

.846 

N 

16 

16 

16 

16 

Self-rated  computer 
knowledge 

Correlation  Coefficient 

.193 

-.246 

.053 

1.000 

Sig.  (2 -tailed) 

.474 

.358 

.846 

N 

16 

16 

16 

16 

Figure  2.  Correlation  significance  table  showing  relationship  between  dependent  variable 
(! TotalEventsCount )  and  nominal/ordinal-level  variables  ( Gender ,  Job,  Self-rated  computer 
knowledge).  There  were  no  significant  correlations.  Spearman’s  rho  test  was  used  to  analyze 
correlation  between  the  non-parametric  variables  (Polit  &  Beck,  2012). 
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Frequency  distribution 

Analysis  of  frequency  distribution  revealed  some  providers  changed  a  limited  number  of 
key  events  from  their  pre-programmed  settings  (Figure  2).  Specifically,  more  than  50  percent  of 
the  time  providers  changed  the  pre-programmed  settings  of  the  events:  TV  in  situ  (content),  TOTS 
[Turned  Over  To  Surgeon,  a  data  point  particular  to  military  operating  rooms]  (time), 
Intravenous  perioperative  antibiotics  (Med,  Dose  /  Declined)  (time  and  content),  Manual  vital 
signs  [the  institution’s  event  for  an  immediately  postoperative  anesthesia  note,  entered  in  the 
PACU]  (time  and  content),  Estimated  blood  loss  (time  and  content),  Urine  output  (time  and 
content),  among  others  (Table  1;  Figure  3).  Providers  altered  the  setting  of  the  surgical  “time 
out”  safety  pause  about  30  percent  of  the  time,  from  the  programmed  one  minute  prior  to  start  of 
surgery  setting,  to  the  time  it  actually  occurred  during  that  case.  About  one  in  five  providers 
altered  the  programmed  time  and  content  of  laryngeal  mask  airway  notes  to  remove  the 
“[endotracheal  tube]  intubation”  notation  and  leave  the  “supraglottic  airway”  event.  Changes  to 
other  events  occurred  less  than  10  percent  of  the  time.  Several  events  were  never  altered  by 
providers,  or  altered  so  rarely  as  to  be  likely  an  anomalous  occurrence  or  mistake. 
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Figure  3.  Events  changed  by  providers  as  a  percentage  of  cases  counted.  Percentages  are  on  the  y-axis.  Events  changed  are  on  the  x- 


axis. 
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Discussion 

Current  information  on  the  accuracy  of  electronic  anesthesia  records  is  that,  generally, 
they  are  more  accurate  than  handwritten  records,  although  methods  to  ensure  complete  and 
correct  records  are  evolving,  and  further  research  is  necessary.  Because  voluntary,  manual 
reporting  of  critical  incidents  and  outcomes  lags  behind  in  completeness,  some  sort  of  secondary 
audit  is  probably  necessary  (Anderson  &  Merry,  2015;  Simpao  et  al.,  2012).  Furthermore,  data 
from  AIMS  (and  all  EHRs)  must  be  validated  in  order  to  be  useful  in  outcomes  research  (Kadry, 
Feaster,  Macario,  &  Ehrenfeld,  2012),  because  changing  practice  based  upon  incorrect  data 
would  be  detrimental  instead  of  helpful. 

As  noted  in  the  survey  results,  Fort  Benning  providers  tended  to  be  positive  about  the 
accuracy  and  usability  of  their  Innovian  system.  However,  the  results  of  the  event  counts  show 
that  providers  tended  to  change  only  those  events  that  were  required  (the  antibiotic  prophylaxis 
noted  is  mandated  by  the  DoD),  specific  legal  requirements  (the  “time  out”),  or  anesthesia 
procedures,  such  as  an  intubation  note.  Stock  events  -  those  typically  relegated  to  a  checklist 
format  on  a  paper  form  -  were  typically  left  in  place.  These  included  notations  about  the  start  of 
anesthesia  care  time,  which  is  of  importance  in  civilian-sector  insurance  billing,  but  less  so  in  a 
military  environment.  Other  items  commonly  left  unchanged  were  automated  charting  of  bite 
block  insertion  and  airway  suctioning  prior  to  extubation. 

Discussion  with  providers  after  data  collection  was  completed  yielded  the  idea  that  such 
items  were  not  necessarily  time  critical,  and  therefore  simply  need  to  be  in  the  chart,  somewhere, 
as  appropriate.  Other  items  typically  ignored  or  forgotten  were  documentation  of  outcomes,  such 
a  laryngospasm.  Critical  events  usually  require  all  of  the  provider’s  attention  and  energy,  and 


charting  is  typically  after-the-fact  (Ehrenfeld  et  al.,  2011).  Frequently,  this  leads  to  a 
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reconstruction  of  the  event  from  memory.  In  the  case  of  Innovian  and  the  DoD,  outcome  charting 
must  be  digitally  signed  by  the  provider.  Signing  a  note  requires  calling  up  that  menu  item  (at 
least  one,  possibly  three  mouse  or  key  clicks),  typing  a  prompt  for  the  provider’s  name,  and 
typing  in  a  complex  password  of  at  least  16  digits,  including  upper  and  lower  case  characters, 
numbers,  and  special  characters,  which  the  DoD  mandates  be  changed  every  60  days.  All  of  the 
requirements  together,  coupled  with  the  patient-focused  nature  of  the  event,  makes  it  unlikely 
that  such  charting  is  completed  until  after-the-fact. 

The  frequency  distribution  of  events  changed  from  the  preprogrammed  templates  implies 
that  providers  as  a  professionally  trained  group  place  more  importance  on  accurate  charting  of 
certain  key  events,  than  on  keying  standard  entries  usually  relegated  to  a  checklist  on  paper 
charts.  The  lack  of  distinction  between  age,  experience,  and  practice  implies  that  this  importance 
is  more  of  a  standard  in  the  community  than  a  personal  preference.  One  goal  of  presenting  this 
hypothesis,  regarding  familiarity  with  computers,  was  the  idea  that  electronic  charting  might  be 
improved  if  provider  felt  comfortable  in  the  digital  environment.  Several  studies  (Avidan  & 
Weissman,  2012;  Ehrenfeld  et  al.,  2011;  Simpao  et  al.,  2012;  Wilbanks  et  al.,  2013)  noted  that, 
when  comprehensive  reporting  of  all  events  during  an  anesthetic  was  lacking,  human  error  was 
primarily  to  blame.  It  was  a  premise  of  Avidan  and  Weissman  (2012)  that  charting  completeness 
was  greatly  improved  by  making  data  entry  easier  for  the  providers.  This  led  to  the  question  of: 
What  if  providers  who  were  more  facile  with  computers  were  better  at  charting  electronically?  It 
was  this  anecdotal  observation  that  was  the  genesis  for  this  project.  During  data  collection,  it  was 
noted  that  some  less  computer  savvy  providers  tended  to  chart  these  basics,  and  only  these 
basics.  However,  because  the  overall  pattern  tended  toward  the  same  important  events,  and  likely 


the  low  number  of  cases  reviewed,  there  was  no  statistical  distinction  at  this  level.  Wilbanks  et 
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al.  (2013)  found  near  perfect  compliance  at  98.38  percent  with  providers  charting  according  to 
the  minimum  requirement  of  the  AANA  standard  by  focusing  on  those  minimum  important 
standards.  In  that  study,  the  primary  source  of  inaccurate  charting,  at  45  percent,  was  altering  the 
preprogrammed  gas  flows,  which  were  not  recorded  by  automation,  to  reflect  reality.  Again,  at 
busy  moments  during  a  complex  pediatric  induction  of  anesthesia,  providers  perhaps  had  less  (or 
felt  they  had  less)  time,  and  subsequently  failed  to  return  to  the  chart  to  correct  their  errors.  In  the 
Fort  Benning  project,  automated  templates  ensured  that  all  of  the  required  data  elements  were  on 
the  chart,  but  could  not  ensure  that  the  course  of  the  anesthetic  took  place  as  specified,  event  by 
event,  as  it  rests  in  the  electronic  record.  To  be  fair,  when  paper  charts  are  reconstructed  after  the 
fact,  the  same  factors  of  memory,  stress,  and  complexity  would  come  into  play  (Shibutani, 

1990).  As  suggested  by  (Ehrenfeld  et  al.,  201 1),  many  providers  put  aside  charting  in  favor  of 
focusing  attention  on  the  patient  for  some  important  activities. 

The  DoD  attempts  to  standardize  events  in  order  to  facilitate  data  mining  of  the  large 
database  generated  by  many  facilities  worldwide  entering  their  cases  into  a  common  pool.  Using 
a  required  item  (such  as  for  antibiotic  prophylaxis  or  an  outcome  for  reintubation  of  a  surgical 
patient)  would,  theoretically,  create  a  common  label  for  these  events,  which  could  be  referenced. 
However,  the  Innovian  system  still  relies  upon  free-text  comments  for  the  details  of  these  events. 
Further,  the  list  of  more  than  500  events  and  outcomes  is  not  searchable  using  context-sensitive 
terms,  only  by  knowledge  of  the  exact  phrasing.  Providers  discussed  this  aspect  of  Innovian  use, 
noting  that  they  used  only  certain  stock  (preprogrammed  by  the  template)  events,  and  used  free- 
text  narrative  comments  for  everything  else.  This  defeats  the  intention  of  DoD  to  data  mine  these 
records  and  was,  also,  part  of  the  reason  for  developing  this  project. 
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Ensuring  validity  of  human-to-human  transfer  of  information  requires  a  common 
dictionary  of  definitions  (Kohli  &  See-Lin  Tan,  2016).  Diagnoses,  drug  names,  diseases, 
treatments,  assessments  -  all  medical  terminology  must  have  the  same  meaning  for  all 
participants  in  the  system  of  standardization  and  sharing  (Braa  et  al.,  2009;  Kohli  &  See-Lin  Tan, 
2016).  Several  standardization  organizations  have  been  developing  the  healthcare  vocabulary  for 
such  communication,  such  as  the  Health  Level  Seven  (HL7),  which  is  used  to  import  information 
from  the  surgical  scheduling  software  each  morning  into  the  Innovian  system. 

Conclusion  and  Implications 

Though  the  hypothesis  of  this  particular  project  was  not  supported,  research  from  the 
review  of  literature  supports  further  development  of  AIMSs  and  EMRs  to  improve  functionality, 
accuracy  and  completeness.  A  further  pilot  study  of  selected,  clinically  significant  events, 
conducted  with  more  providers  and  more  cases  (perhaps  using  automated,  rather  than  manual, 
tallying  for  a  more  comprehensive  analysis),  and  using  more  sensitive  statistical  tests,  might 
yield  significant  correlation.  A  more  important  focus  should  be  the  development  of  more  user- 
friendly,  less  time  consuming  graphical  user  interfaces  (GUIs)  -  such  as  voice  commands  -  to 
facilitate  accurate,  complete  charting  on  electronic  anesthesia  records  (Alapetite,  2008;  Ehrenfeld 
et  al.,  2011).  More  research  into  usability,  designs  to  facilitate  better  usability  and  completeness, 
examination  of  workflows  and  ergonomic  processes  required  to  enter  data,  and  common 
processes  and  terminology  to  facilitate  data  sharing  are  necessary  to  advance  this  nascent  field  of 
medical  informatics.  The  prospect  of  very  large  datasets  could  answer  research  questions 
previously  too  complex  to  address  using  randomized  controlled  trials  or  meta-analyses,  and  offer 
insight  for  improved  quality  control  (Ehrenfeld,  2009;  Hurrell  et  al.,  2012).  This  opinion  is 
supported  by  Anderson  and  Merry  (2015);  and  Kohli  and  See-Lin  Tan  (2016).  Lack  of 
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completeness  may  invalidate  the  usefulness  of  these  datasets  in  research.  AIMS  require  more 
research  in  usability,  design,  workflows  and  ergonomics,  and  common  terminology  to  facilitate 
data  sharing.  There  is  no  standardized  method  to  examine  data  quality  (Weiskopf  &  Weng,  2013) 
or  achieve  interoperability  between  information  systems  (Hurrell  et  al.,  2012;  Kohli  &  See-Lin 
Tan,  2016).  This  lack  will  likely  hinder  further  study  of  AIMS  software.  Furthermore,  a  lack  of 
accuracy  and  completeness  of  these  records  calls  into  question  their  validity  as  a  research  tool 
(Anderson  &  Merry,  2015;  Kohli  &  See-Lin  Tan,  2016). 

Three  studies  (Braa  et  al.,  2009;  Hurrell  et  al.,  2012;  Walsh  et  al.,  2009)  outline  strategies 
to  achieve  AIMS  EHR  interoperability.  The  elements  of  interoperability  described  were  system- 
to-system  communication  (syntactic),  provider-to-provider  communication  (semantic),  and 
flexible  development  of  standardized  data  sets  in  an  evolving  healthcare  environment.  These 
strategies  should  be  pursued  by  the  healthcare  informatics  community  and  the  industry  that 
provides  these  products. 

Financial  disclosure.  The  author  has  no  financial  interest  in  any  anesthesia  electronic 
management  software ,  or  other  electronic  medical  record  developer  or  company. 
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Comparison  of  physiologic  variables  charted  in  handwritten  and  electronic  records  (van 
Schalkwyk,  Lowes,  Frampton,  &  Merry,  2011). 
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